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A Content Strategy for Mobile Devices 


What Now? 



n the fall of 2013, Apple CEO Tim Cook announced that the number of 
apps in Apple's iTunes store had reached one million (The Verge, 2013). 
With more than 230 new apps of all kinds submitted to iTunes every day, 
the number of apps identified as "educational" in the iTunes store has 
now grown to a staggering 122,000 (148Apps, 2014). 


Now put these numbers in the context of children's access to and, presumably, 
use of mobile devices and apps at home, access which has grown explosively over 
the last several years. According to a recent study published by the Brookings Insti- 
tute (West, 2013), 18% of children in Grades l<-2 have access to a smartphone, and 
26% have access to a tablet device; 45% of third through fifth graders have access to 
smartphones, and 48% have access to tablets; 65% of sixth through eighth graders 
have access to smartphones, and 52% have access to tablets; 80% of ninth through 
twelfth graders have access to smartphones, and 45% have access to tablets. These 
percentages— up dramatically from just a few years ago— continue to grow daily. 


The use of mobile devices presents a similar trajectory in U.S. schools. According 
to PBS LearningMedia (2013), tablets and e-readers saw the biggest increase among 
technology platforms available for classroom instruction between 2012 and 2013. 

In February 2013, more than one third (35%) of teachers said they had access to a 
tablet or e-reader in their classroom, up from 20% a year previously. Among teach- 
ers with access to tablets, 71% cited the use of educational applications as the most 
beneficial for teaching. Currently, about 10 million tablet devices are being used in 
schools and other educational settings. 

Whenever a large purchase of mobile devices is made for education, the very 
next question asked is, "Now what?" The concerns up and down the hierarchy of the 
education system, as devices are implemented, vary. Teachers are concerned about 
the practical issues of using devices with students and selecting content that helps 
students learn. School and district administrators want to ensure that their purchas- 
ing dollars have been used meaningfully and that the devices are being used regu- 
larly in the classrooms. And state department of education personnel want to know 
that the implementation of new technologies is helping the students of their state 
perform at grade level, prepare for college and careers, perform well on standardized 
tests, and compete favorably with students from other states. 


But in order for all of these levels of concern to be addressed successfully, the 
fundamental question of "Now what?" needs to be addressed immediately when the 
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devices arrive at an individual school or classroom. It is the planning for and using of 
the devices as part of an overall approach to the curriculum, locally, that will deter- 
mine how the interests of different stakeholders are satisfied. ..or not. 

This practice guide is intended to help stakeholders in the following ways: 

• To assist teachers, curriculum and technology specialists, and administrators 
plan and implement the use of mobile devices and content in schools and 
districts. 

• To help administrators understand the content and implementation planning 
that is required for a successful mobile device implementation before they make 
a large expenditure on hardware, lest that outlay of money be a waste in the 
absence of a larger strategy. 

• To give state education agencies insight into what happens in local schools' 
and districts' planning for content and curriculum as part of a mobile device 
implementation. 

The following assumptions have been made in writing this practice guide: 

• A purchase of mobile devices, a decision about what type of mobile devices to 
buy, or a decision to implement a bring-your-own-device (BYOD) program in the 
school or district has already been made. 


]ust beginning the process of mobilizing classrooms? Here are a few web 
resources to help you get started: 

Creating a School Technology Plan 

• National Center for Technology Planning: http://www.nctp.com 

• Universal Service Administrative Company: http://www.usac.org/sl/ 
applicants/stepOl/ 

• North Central Regional Education Laboratory: http://www.ncrel.org/sdrs/ 
areas/issues/methods/technlgy/te300.htm 

Deciding on Devices for a School 

• eSchool News: http://www.eschoolnews.com/2014/03/04/ 
choosing-right-mobile-devices-schools/ 

• K12 Digital Decisions: http://k12digitaldecisions.com 

Considering a BYOD Plan 

• EdTech Magazine: http://www.edtechmagazine.com/kl2/sites/edtechmaga- 
zine.com.kl2/files/l 1 1331-wp-kl2-byod-df.pdf 

• K-12 Blueprin: http://www.kl2blueprint.com/byod 

• TeachThought: http://www.teachthought.com/ 
technology/20-byod-resources-for-2 1 st-century-school/ 
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• A technology plan that addresses the technical aspects of a mobile device 
implementation— such as sufficient broadband, networking of devices, and 
device management— has already been established. 

• The reader is at the point of asking, "Now what?" in addressing content and cur- 
riculum in the implementation of mobile devices in individual classrooms. 

So let's begin with preparing yourself, your classrooms, and your learners. 


Preparing to Select Content 



efore selecting particular digital content to use on the devices in the 
classroom, the teacher or technology director must familiarize herself 
with some of the technical aspects of the devices she will be using. 
These technical aspects will provide information that will help in 
selecting appropriate digital content that is compatible, technically, 
with the devices a school has purchased. 


Technical Issues 

Hardware features and functions (and perhaps limitations) will influence what 
you and your learners will be able to do with content and how you do it. This first 
section is about getting to know your device(s) and some background information 
you'll need to know as you prepare to build a content strategy. 


What Platform Are You Using ? 

Before even thinking about content, make sure you know what platform (also 
known as "operating system" or "OS") your device uses and where to shop for con- 
tent (known as "apps," short for "applications") for that platform. The technology 
director at your school or district can help you with this. The four major platforms 
and their online "stores" are: 

• iOS-iTunes: https://itunes. apple. com/us/genre/ios/id36?mt=8 

• Android - Google Play: https://play.google.com/store/apps7hNen 

• Chrome - Chrome Web Store: https://chrome.google.com/webstore/category/ 
apps 

• Microsoft Apps for Windows: http://windows.microsoft.com/en-us/windows-8/ 
apps#Cat=tl 

Until recently, iOS-based apps comprised the largest number of educational apps, 
but iOS numbers have been eclipsed as other platforms' offerings have grown. In 
March 2014, iTunes reported having more than 65,000 educational apps for iOS, but, 
a month earlier, the number of education apps for Android had surpassed 77,000 
(AppTornado GmbH, 2014). Chrome and Microsoft are newer entrants to the mobile 
device industry, and estimates of the number of educational apps in those stores 
were not available at the time of this writing. 


If your school or district is implementing a bring-your-own-device (BYOD) pro- 
gram, the issues are more complex. You are faced with either choosing apps that are 
cross-platform (i.e., built to run on more than one platform) or with choosing different 
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apps for different platforms. Either course may not be easy. In the first case, there are 
still relatively few educational apps that are developed for multiple platforms; in the 
second case, search time will be multiplied by the number of platforms you are try- 
ing to accommodate. We'll talk more about selecting apps shortly, but in the case of 
BYOD, make sure that you know all of the platforms you must accommodate. 


What Devices Are You Using? 

Even if your school or district has adopted a particular operating system within 
a BYOD program, you may still be confronted with a wide range of possible devices. 
With iOS, there are only a few devices, all manufactured by Apple (iPhone, iPad); but 
if you are using Android, for example, you may be using a device that is manufac- 
tured by Sony, Samsung, HTC, Amazon (i.e.. Kindle), Barnes and Noble (i.e.. Nook), 
Google, and many others. Why does the device and its manufacturer matter? Well, 
when developers create apps, they are usually optimized for particular devices or 
screen sizes. For example, an app that is optimized for an iPhone will not display 
as nicely on an iPad as will an app that is optimized for the iPad. And an app that is 
built specifically for the iPad won't run on an iPhone at all. Similar types of incompat- 
ibilities exist among the different manufacturer's Android devices. So you'll want to 
be aware of the specific devices used by your students as you start considering the 
apps that you want to use. When you shop in an app store, you will find a list of all of 
the devices for which a given app is optimized. Make sure that an app runs on your 
device(s) before purchasing it. 


What Operating System is Running on Your Device(s)? 

If you are using an Apple device, determining the operating system (OS) is fairly 
straightforward because Apple only has one operating system, iOS, and it is updated 
periodically; each version is referred to by number, for example, "iOS 7." When 
selecting an app, the user should choose an app that supports the current operating 
system running on the device or an earlier version of the operating system that is still 
supported by the device (for information on how to check the current iOS running on 
your device, go to http://support.apple.com/kb/ht2188). The same operating system 
runs on iPhones and iPads, but apps may be optimized for one device or another. 
Apps that are optimized for iPad cannot run on iPhones; apps that are optimized 
for iPhone, however, do run on iPads (the graphics are optimized for the smaller 
screen size so they may appear grainy when displayed on the iPad at approximately 
twice the optimal size). Many developers optimize for both the iPad and the iPhone. 
Updates to the iOS operating system are free and the user is alerted, on the device, 
when an update to the operating system is available. Updates do not happen auto- 
matically, but must be initiated by the user. 

If you are using an Android device, determining the OS is more complicated 
because different device manufacturers support different versions of the AndroidOS. 
So, you will hear about Android operating systems such as "Ice Cream Sandwich," 
"Jelly Bean," and, most recently, "KitKat." You'll need to check and see which version 
of the operating system your particular device supports. If you are an Android user 
who shops in the Google Play store, once you synchronize your device with the store, 
the device will automatically tell you if the app will work on your particular device. 
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Purpose of the App to be Selected 

After you've got the essential technical issues under your belt, the next set of 
questions has to do with why you're looking for an app. It's best to know what you're 
trying to accomplish with an app and what functions or coverage you're looking 
for before going into the app store, where you'll undoubtedly be drawn under the 
spell of all of the really cool-looking apps that are available. In fact, at a recent event 
held by Google that I attended, teachers, parents, and kids alike said that one of the 
major factors in their choosing an app was the attractiveness of its icon. It's sort of 
the technology equivalent of going to the grocery store when you're hungry! 

Educational apps fall into four basic categories based on their purpose: instruc- 
tion, creativity, tools, and books. Which type of app you select will depend on what 
you expect the learner to do with an app or what you expect an app to do for a 
learner. Here's the breakdown: 


Instruction 

Apps devoted to instruction aim to teach particular skill 
sets to learners, establishing initial, simple performances 
and then building increasingly more complex performances. 
These apps explicitly teach new skills to learners or 
learners gain expertise with those skills. The 
of subject-specific apps fall into this category. Some 
well-known examples include DragonBox Algebra 5+ 
(WeWantToKnow AS, 2014), Learn with Homer (Ho 
Learning, Inc., 2014), and LetterSchool (Sanoma 
Netherlands B.V., 2013). 


Creativity 

Also known as "productivity apps," creativity 
apps allow learners to extend and apply their 
skills to novel situations and make their own 
"stuff." These apps do not provide explicit 
instruction; instead, they provide a virtual 
sandbox in which learners create their own experi- 
ence, using skills that have been established previously. 

Creativity apps enable learners, for example, to make their own 
e-books, tell a story, assemble a presentation, draw pictures, compose 
music, or make movies. In these apps, the learning objectives are less specific 
than those in instructional apps, but learners still must have the necessary entry- 
level skills for the given creativity app. For example, to use an e-book creation app, 
a learner would need sufficient writing, reading, spelling, and grammar skills to 
use it meaningfully. Because learners can produce such a wide array of products 
using creativity apps, these apps do not provide performance evaluation, which is 
left to the teacher. Some well-known examples of creativity apps include Explain 
Everything (MorrisCooke, 2014), Toontastic (LaunchPad Toys, 2014), and Story Cre- 
ator (Innovative Mobile Apps, 2013). 
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Tools 

Tool apps are those that learners can refer to for more information relevant to 
whatever topic they happen to be studying. These apps might include reference 
tools, such as dictionaries, calculators, and periodic tables. They might also provide 
simulations in which learners can observe animated concepts and phenomena at 
work, such as the scientific principles of acceleration and gravity. The key with apps 
of this type is that they don't evaluate learner interactions for correctness or qual- 
ity. Instead, most or all of the information is presented passively to the user. Some 
examples include World Book: This Day in History (Software MacKiev, 2011), GoSky- 
Watch Planetarium (GoSoftWorks, 2013), and PBS Kids Video (PBS Kids, 2014). 

Books 

This category probably seems pretty obvious. Many book apps are just digitized 
versions of books. But there's also a subcategory known as "interactive" e-books. 
What qualifies as "interactivity" can vary widely, from simple scrolling from page 
to page to more meaningful tasks such as identifying of objects on the screen or 
answering simple questions. There's such a range that the descriptor "interactive" is 
no guarantee an app in this subcategory will prompt educationally relevant learner 
responses. 

Now that you know the four basic categories of apps, you can decide what you 
expect of an app. Do you need a learner to acquire a new set of skills? Does a learner 
already have a set of component skills and now you want her to extend and apply 
those skills by creating something new? Or perhaps your students just need a refer- 
ence tool or a book. Whatever the case, identifying the learning objective that will 
be serviced by an app will facilitate your search. 


Selecting Content 

f you've ever shopped for and used apps, whether for use in an 
educational setting or just for your personal use, then you know that 
not all apps are created equal. And none of us likes wasting time with 
apps that don't do what we hope they will do. ..or what the app store 
description says they'll do! So, this section provides a "checklist" 
of things that will help identify high-quality apps. The citations 
accompanying the items in the checklist refer to research studies supporting 
the importance of that feature. Bear in mind that the more specific the learning 
objective that you're trying to address, the more closely the app will need to be 
evaluated to make sure that specific skill is taught. For example, if you want an 
app that will help a learner solve algebraic equations, that is a more specific skill 
than is creating a movie or reading a book. Plan on spending more time doing a 
more complete evaluation for the specific skills. Below, I have listedthe features, by 
category, that indicate that an app is of high quality. 
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Instructional/Skill Acquisition Apps 

• Feedback: The app provides immediate feedback following both correct and 
incorrect answers; the feedback is noticeably different for correct vs. incorrect 
answers (e.g., Azevedo & Bernard, 1995; Van Houten, 1984). 

• Adaptive Difficulty: The difficulty of the material increases and decreases auto- 
matically, depending on the learner's performance. This adaptation happens 
on-the-fly, without the learner or an adult needing to change the settings on the 
app (e.g., Tsai, Kinzer, Hung, Chen, & Hsu, 2013). 

• Mastery-based: The app requires the learner to achieve mastery of the current 
skill set before being allowed to progress to the next level (e.g., Kulik & Kulik, 

1990). 

• High Numbers of Relevant Responses: The app provides plenty of opportuni- 
ties for the learner to practice the skills related to the learning objective (e.g.. 

Greenwood, Delquadri, & Hall, 1984). 

• Clear Learning Objectives: A skill that a teacher or parent can observe a learner 
doing is clearly described in the app or app description. This means that what 
the learner does and the conditions under which the attempt occurs must be 
defined clearly enough so that two people can agree on whether or not it is 
happening (e.g., Ericsson & Lehmann, 1996). 

• Performance Reports With Actionable Data: Learner performance data are 
reported with enough detail for a teacher or parent to target problem areas. For 
example, if the app targets phonics, the report should include details of accura- 
cy with specific letter-phoneme correspondences, not a simple percent correct 
for the whole skill category (e.g., Wayman, 2005). 

• Usability: The app should be easy to use, with simple instructions (either textual 
or graphic) of how to interact with the interface (e.g., Gerhardt-Powals, 1996). 

Additionally, the images and sounds included in an app should be relevant to 
the learning activity, not distracting for the learner (Mayer, Heiser, & Lonn, 

2001). And finally, the reading level of the app should be appropriate for the 
lowest age of learner identified by the app developer (e.g., Hanna, Risden, Czer- 
winski, & Alexander, 1998). 

Creativity/Skill Extension and Application Apps 

• A high-quality app includes clear instructions for the user and, if necessary, tuto- 
rials (e.g., Gerhardt-Powals, 1996). 

• The layout of the screen can be used effectively on a mobile device, with its 
small screen size. 

• The product created by the learner can be exported via common file formats 
(that will allow the product to be used in other ways— e.g., pdf, doc, ppt) and 
saved. 

• Files— such as images, sounds or movie clips— may be imported into the app. 

• The prerequisite skills that the learner will need to use the app effectively are 
clearly described. 

• Preferably, the app allows students to collaborate with each other. 
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Tools 

• The information included in the tool is accurate. 

• The tool is intuitive and easy to use (e.g., Gerhardt-Powals, 1996). 

• The information is presented clearly. 

• The reading level is appropriate to the age of the learner (e.g., Hanna, Risden, 
Czerwinski, & Alexander, 1998). 

Books 

• The reading level of the book is appropriate to the age of the learner (e.g., 
Hanna, Risden, Czerwinski, & Alexander, 1998). 

• The content of the book is appropriate for the age of the learner. 


How NOT to Select Content 



o far, this practice guide has identified what to look for in apps. Almost 
as important as features to look/or are selection criteria to ovoid when 
choosing apps for your learners. 


Familiarity 

As consumers, we often buy products from companies with brands that we 
recognize. In the educational app market, those recognizable brands are often large 
educational publishers and media companies. But just because these companies 
make great textbooks, movies, and television shows doesn't necessarily mean they 
will make great educational apps. Apps produced by these companies tend to be 
very visually appealing, oftentimes with familiar characters from other media, but 
are hit and miss with respect to instructional quality. 


Price 

How many of us have shopped for a product and bought the one that was priced 
"kind of in the middle," on the one hand, because we didn't really want to spend 
to buy the most expensive option, and, on the other, because the cheap or free 
options couldn't possibly be good. I know that I'm guilty of that. But in educational 
apps, price doesn't predict quality. At Balefire Labs, we recently ran some analyses 
of the more than 1,300 app reviews that we've completed. When we ran a correla- 
tion between price and quality score, we found that there is no relationship between 
price and quality (r = 0.0547) (Balefire Labs, 2013, October 29). This doesn't mean 
that you shouldn't spring for apps that you pay for. It just means that you can't 
assume that an expensive commercial app is any better than a less expensive one, or 
that either is better than a free one. 


The Developer’s Description or User Reviews 

If you've been to any of the app stores lately, you've probably noticed something: 
Pretty much all of the apps claim that they will teach your kids. And if you've down- 
loaded any variety of educational apps, you know that's simply not so. And those 
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"user reviews" that appear below the app description have some problems with 
objectivity. We would expect that developers' friends and families would post very 
flattering reviews of a product. But what I was surprised to learn— call me naive— is 
that sometimes competitors try to sabotage each other by posting negative reviews 
in the app stores. It seems to be a widely acknowledged problem in app developer 
circles. The upshot is this: You can't completely rely on the information in the app 
store. 


Popularity 

It's obvious that "Top 10" Charts are based on number of downloads, but for 
some educational apps, there may be good pedagogical reasons for their reaching 
the top of the charts. Of course, my colleagues and I at Balefire Labs got curious 
about that, so we went to one of the app stores and downloaded the top 10 paid 
apps from the education category and I reviewed them all. We found that we could 
recommend only two of the top 10 apps based on their instructional quality (Balefire 
Labs, 2013, August 26). So, it's useful to remember that a lot of people buying an app 
does not necessarily mean it's a good app. 


Colleagues’ Recommendations 

This might be the most tempting option of all. You hear from a friend or col- 
league or reviewer that the kids they work with liked the app, and they're sure your 
kids will like it, too. But here's the thing: Learners like being successful. If kids say 
they like an app, it's likely to be (at least in part) because they have the necessary 
skills to use the app successfully. This doesn't mean that kids only like to use apps 
that are easy for them— quite the opposite. They like to be challenged, just not to 
the point at which they lack the skills to continue to make progress. Although your 
colleague's students may have liked an app, their skill levels could be vastly different 
from those of your students. And that difference will impact your students' enjoy- 
ment. So, unless the app being recommended has adaptive levels of difficulty to 
accommodate all users, be wary. 


Special Considerations 



'm getting close to turning you loose to select apps. ..I promise! But first 
let's talk about special considerations that might be relevant for you. 


Alignment to Standards 

As more states implement the Common Core State Standards (CCSS), you may 
want to select apps for particular standards that you are addressing in your class- 
room. As of now, very few app developers align their apps to CCSS. If this is a prior- 
ity for you, and you don't want to do the alignment yourself, you are probably best 
served by finding a reliable website that curates educational apps and provides CCSS 
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alignment (for which, see the Finding Reliable App Recommendations section later in 
this guide). 


Accessibility 

Depending on the device you are using, certain features may be turned on to 
enhance its accessibility. In the case of iOS devices, for example, some of the acces- 
sibility features include: 

• VoiceOver: When the user touches the screen, VoiceOver reads what is happen- 
ing under the user's finger. 

• Speak Selection: When the user highlights text in any application and then taps 
"Speak," Speak Selection reads the selected text aloud. 

• Zoom: When the user double-taps with three fingers, this built-in magnifier 
zooms in 200 percent. 

Aside from the accessibility settings provided by the devices themselves, a 
remarkably small percentage of app developers include customization capabilities 
to enhance accessibility. Be sure to vet apps completely for accessibility if you have 
students for whom this is a concern. 


Information Safety — Yours and Your Students’ 

When choosing apps, you'll want to consider a few safety features: 

• Does the app include in-app purchases (i.e., purchases that are critical to using 
the current app, such as buying coins or access to a new level) or access to 
the app store to buy additional apps? In some cases, purchases can be made 
without re-entering your login information, so kids may be able to make those 
purchases without your knowledge or permission. 

• Be aware of the links that an app may have to social media. Many apps, includ- 
ing both educational and noneducational, link to online centers that track 
learner scores; many others link to Facebook and Twitter. While access to these 
links may enable students to interact and collaborate with each other, in some 
cases they may allow them to have interactions with strangers (some of whom 
are likely to be adults). Depending on the device and operating system a school 
is using, varying degrees of device "lock-down" are available to limit the abil- 
ity of students to access outside links. Not all teachers and parents opt to lock 
down kids' devices, but most want to know when these kinds of links are pres- 
ent so that they can have discussions with kids about Internet safety. 

• Pay attention to whether or not an app asks to send you "push notifications" 
(i.e., an alert or reminder that is delivered to the device from the developer) or 
collect location information. If a developer collects location information, this 
means that the app company knows the place from which you or your students 
access its app. These location data do not include any personally identifiable 
data, but some are still wary about it, so it's useful to be aware of which apps 
collect it. 

• Finally, one of the biggest concerns for teachers and parents is in-app advertise- 
ments. These are very common in free apps because developers need some way 
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of paying for the development costs, so they sell advertising space. The problem 
is that there have been multiple examples of inappropriate, adult-themed ads 
appearing in children's apps. The approaches used by developers and by adver- 
tising agencies have changed in recent months and these changes seem to have 
addressed the worst violators, such as adult-themed ads. However, there are 
still other ads for items that children might like (e.g., other apps or games) that 
may be a concern for teachers and parents. 


Volume Purchasing 

If you are purchasing multiple copies of an app for devices in your school or 
classroom, you should explore your platform's volume purchasing program. Not only 
will this allow you to buy multiple copies of an app at a discounted rate, but, in most 
cases, these programs also facilitate mobile device management (MDM). (Note: 
MDM is the distribution of applications, data, updates, and configuration 
settings for mobile devices from one centralized location 
within the school; this is completed via a wireless net- 
work.) Here is more information about volume purchasing 
for the four major device platforms in schools today: 

• Apple Volume Purchase/MDM Program: With this 
program, multiple copies of an app may be purchased 
at a discounted rate. Once all devices (which must run 
iOS7 or later) in a classroom or school are enrolled, the 
assignment, revoking, and reassigning of apps to and 
from those devices can be managed centrally. 

• Google Play for Education: Similar to the Apple offering, 

Google Play for Education allows discount, bulk purchasing 
and centralized management of apps that run on Android 
devices. 

• Chrome Devices for Education: Also owned by Google, 

Chrome offers volume purchasing and centralized manage- 
ment of apps that run on Chrome devices via its Admin Con- 
sole, which is accessed by school personnel in charge of the 
mobile device management program. 

• Microsoft Volume Licensing Program: Microsoft offers a vol- 
ume licensing program to educational institutions and an independent mobile 
device management program, for apps that run on the Microsoft mobile 
devices, that must be managed separately. 



Finding Reliable App Recommendations 

One of the most common complaints among teachers and tech directors in 
schools concerns the time-consuming task of finding high-quality educational apps. 
With more than a hundred thousand educational apps available between iTunes 
and Google Play alone, it's no wonder that finding strong instructional apps can 
seem like a daunting, even impossible task. And yet, in spite of how time consum- 
ing the task is, schools and districts all over the country require teachers and tech 
directors to do all of their own app vetting. 
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A better strategy is for schools and districts to rely on educational app review 
services; there are quite a few and they're easy to search for online. Review sites 
provide curated lists of recommended apps, which allow school personnel to spend 
a fraction of the time in identifying potential apps, as they only need to vet just the 
curated list. Different curatorial sites focus on different criteria and have different 
business models. 

Some of the most popular sites include Teachers with Apps (2014), Graphite 
(Common Sense Media, 2013), Appitic (2014), and Balefire Labs (2014; the author 
is the president of Balefire Labs). The curators' websites all offer slightly different 
services: Some are free to the user and supported by advertising and review fees 
(e.g.. Teachers with Apps), some are run by teacher volunteers (e.g., Appitic), others 
are supported by large corporations (e.g.. Graphite), and still others are fee-based 
subscription services (e.g.. Balefire Labs). Appendix I provides examples of reviews 
for DragonBox Algebra 5+ from each of these four curation sites. 

As you shop around for educational app curation sites, here are some questions 
to ask yourself to help determine which site fits your needs: 

• Are the rubrics being used clearly defined and transparent? Can I easily under- 
stand how the reviewers decide whether or not an app is good? Is the rubric 
applied in a systematic way to all apps being reviewed? 

• How did this organization arrive at these rubrics? Are they objective or 
opinion-based? 

• How can I meaningfully use the information provided by an app review when 
I'm working with students? 

• Does this organization charge app developers a fee for completing a review? If 
so, should I be concerned that this is influencing the review the app receives? 

• Does this organization take advertising money from app developers? Again, 
should I be concerned that these monies are influencing the reviews? 

• Once you've tried some of the site's recommended apps, ask yourself, What was 
my experience? Did the recommended apps prove to be useful to my students? 

• Does this site offer alignment of apps to the Common Core State Standards? 

Establishing an Acceptable Use Policy 

Before giving students access to mobile devices, an acceptable use policy (AUP) 
must be established, setting the standards for how students may and may not use 
the devices. An AUP can be set at a classroom, school, or district level, depending on 
the district policy. It is not unusual for students to each sign a copy of the AUP before 
using a device. In elementary classrooms, in particular, the AUP is often posted. 

Each school or district will need to decide what is important in an AUP. AUPs do 
not need to be set in stone; they can be updated and modified as you identify differ- 
ent needs. The following list presents some suggestions— which are not intended to 
be exhaustive— of the kinds of items that are often included in AUPs. 

• Only use apps and programs your teacher has instructed you to use. 

• Do not make any purchases of apps or purchases in apps without the teacher's 
permission. 
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• Do not delete anything that is installed on the tablet without the teacher's 
permission. 

• Always know where the tablet is when you are responsible for it. 

• Keep liquids and food well away from the tablet. 

• Do not put heavy objects on top of the tablet so that you do not break the glass 
screen. 

• Only a soft cloth should be used to clean the tablet screen. 

• Be responsible and make good choices using the tablet. 

• To extend battery life, always turn the tablet off after completing work. 

• Do not expose the tablet to extreme hot or cold. 

• The volume on the tablet should remain off unless headphones are connected 
to the tablet or you have the teacher's permission to turn it on. 

For small children, consider the following for the AUP: 

• I will always carry the tablet with two hands. 

• I will always wash my hands before using the tablet. 

• When the teacher [does X], I will stop what I'm doing, put the tablet down and 
pay attention. 

For older students, consider these requirements in the AUP: 

• Students are not allowed to send, access, upload, download, or distribute offen- 
sive, profane, threatening, pornographic, obscene, or sexually explicit materials. 

• Use of the camera and microphone in the school are strictly prohibited without 
permission from the teacher. 

• Malicious use or vandalism— such as any attempt to destroy hardware, soft- 
ware, or data— is prohibited. 

• Jail breaking— that is, removing limitations placed on a device by its manufac- 
turer— is strictly prohibited. 

Compensating for an App’s Shortcomings 

Often it's hard to find the perfect app for your needs. You might be looking for 
an app that targets a particular age, a particular subject, and even a specific stan- 
dard. But you might not find all of that in a strong instructional app that has all of the 
features that I described earlier. And though initially this limitation might seem like a 
drawback, it is actually a learning opportunity that helps teachers enhance their app 
integration skills. Imagine this scenario: A teacher approaches a technology coordi- 
nator and asks her to purchase a particular app. Together, the teacher and tech coor- 
dinator check their preferred curation site for more information and a review of the 
app. Perhaps they discover that the app does not include any error remediation, the 
reading level is too high for the student who needs it, and the app doesn't include 
learner support. This doesn't necessarily mean that the app should be dismissed for 
possible use. But what it does mean is that now the teacher and the tech coordina- 
tor have the chance to discuss how the teacher might need to augment the student's 
use of the app. If the teacher says to the technology coordinator, "Well, those things 
don't matter to me that much because I plan to sit with the student and help him 
use the app. I'll provide the help that the app doesn't provide," then a less optimal 
app may work just fine. The key is to examine what a given app lacks and make a 
plan for best use of that app, taking those shortcomings into consideration. 
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Conclusion 



reating a content strategy can be a daunting task for anyone 
implementing mobile devices in education. But having a strategy to 
prepare for, select, and use content with mobile devices is critical to the 
goals of all of the stakeholders in the education system implementing 
such devices. The principles and recommendations covered in this 
guide will help get you started. 
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Appendix 1: Screenshots — Sample App Evaluations 
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Balefire Labs (www.balefirelabs.com) 



DragonBox Algebra 5+ 

Publisher: WeWantToKnow AS 
Platform: iOS, Android 
Version: 1 .0.0 
Price: $5.99 

Age: lOyrs, 11 yrs, 12yrs, 13yrs, 14yrs, 15yrs, 16yrs, 17yrs, 18 
yrs 


This app requires learners to move mathematical objects on the screen to 
solve algebra problems. 


Balefire Labs Review 

o 

Instructional Design 

Q Feedback for Correct 
Responses 

Q Error Feedback 

Q Adapting Difficulty 

O Error Remediation 

O Mastery-Based Instruction 

Q Frequent, Meaningful, Learner 
Interaction 

Q Clearly-Stated Learning 
Objectives 


Usability Design 

Q Relevant Screen & Sound Use 

Q Learner Support Available 

O Easy-to-Use Interface 

O Age-Appropriate Reading Level 

© Performance Reports with 
Actionable Data 


Common Core State Standards Alignment: 

Mathematics 

» 4.0 A. A. 2 - Multiply or divide to solve word problems invoMng multiplicative comparison, 
e.g., by using drawings and equations with a symbol for the unknown number to represent 
the problem, distinguishing multiplicative comparison from additive comparison. 

» 6. NS. C. 6a - Recognize opposite signs of numbers as indicating locations on opposite sides 
of 0 on the number line; recognize that the opposite of the opposite of a number is the 
number itself, e.g., -(-3) = 3, and that 0 is its own opposite. 

» 6.EE.A.2 - Write, read, and evaluate expressions in which letters stand for numbers. 

(continued) 
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» 6.EE.B.5 - Understand solving an equation or inequality as a process of answering a 
question: which values from a specified set, if any, make the equation or inequality true? Use 
substitution to determine whether a given number in a specified set makes an equation or 
inequality true. 

» 7.NS.A.1a - Describe situations in which opposite quantities combine to make 0. 

* 7.EE.B.4 - Use variables to represent quantities in a real-world or mathematical problem, and 
construct simple equations and inequalities to solve problems by reasoning about the 
quantities. 

• 8.EE.C.7 - Solve linear equations in one variable. 


O Less 


Publisher Description 

NOW EVERYONE CAN LEARN DIFFICULT MATH AND HAVE FUN 

SOLVING ALGEBRA EQUATIONS AFTER JUST ONE HOUR OF PLAYING 

(Note: DragonBox+ is not an upgrade or extension pack. It is the full DragonBox game plus 
1 00 extra levels). 

Many children today struggle with mathematics while other children want greater challenges. 
DragonBox is a revolutionary game that will make your child enjoy learning mathematics 
while progressing at his or her own pace. 

Our intention has been to create a game that children experience as a real game that is 
actually fun, but where they after one hour of playing, they will be able to solve mathematic 
equations that they never thought they would be able to solve so quickly before. 

The DragonBox game has been tested in schools in France and in Norway, and the results 
clearly demonstrate increased understanding of basic mathematics and the improved ability 
to solve equations. More importantly, it shows a significant positive change in childrens 
attitude towards learning mathematics. 

DragonBox is taking Game Based Learning to a new level and will contribute to demystifying 
algebra in such a way that children at all levels will start to enjoy and embrace learning 
mathematics. 

It is a great game for parents to play along with their kids and freshen up their own math 
skills. 

DragonBox+ contains an additional 100 standard equations to help your child become a real 
master of algebra. 

The game is in English, French, Norwegian, Spanish and Swedish. 
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DragonBox Algebra 5+ - the Revolutionary 
Algebra App 

BY REBECCA SIMON ■ OCTOBER 18, 2012 • INTERMEDIATE MATH, MATH, TWA CERTIFIED. TWA PICKS ■ 5 COMMENTS 


Download on th# 

• AppStore 


Dragon Box, by WeWantToKnow AS - 

I vividly remember the days long ago, 
when I was in middle school learning 
algebra. There was "y", there was "x", 
there was "x+y=5", there was ”l/x + 

2/y = H", and so forth. Most middle school students would not 
find learning pre-algebra or algebra that exciting. On the 
contrary, they might find it quite boring. Sitting and learning a 
bunch of letter and number combinations, which have no 
meaning to the students whatsoever, is just not appealing or engaging. In addition, the 
equations and formulas being taught are abstract; they are not something concrete 
that the students can relate to. It is the teacher's job to teach the material in a way the 
students can relate to... to make it meaningful to them. 





As we are moving further into the 21st century, students are expecting that their 
learning include digital devices. In addition, it is becoming increasingly common that 
the content being taught is presented in a way that includes online gaming, 
simulations, and/or augmented reality. What better way to teach abstract content, such 
as algebra, than to teach it with games?!? Teaching with games motivates and engages 
students. 


(continued) 


19 






A Content Strategy for Mobile Devices 


Dragon Box is highly beneficial for students in middle school. Although, those who are 
younger and even adults who are "young at heart" will enjoy it as well. I gave my son, 
who is in seventh grade, a chance to field-test the Dragon Box and I played it as well. 
We both enjoyed it immensely! Even more so, it was quite addicting! Rather than 
giving a problem such as "x+y=5" to solve, the app game starts at a very basic level. 
For instance, instead of starting with solving equations and problems, the game first 
teaches the user how to discriminate between two characters and to separate them. It 
affords the opportunity to practice the same steps that are involved in Algebra. 
However, they are presented in such a exciting way, that the user would never know it. 
This includes such steps as applying the same strategy to both sides of the equation, 
looking for like terms, and changing whole numbers into fractions. It also gives step by 
step directions early on and directs the user as how to proceed. Instead of characters 
such as *x" and "y," the game uses avatars and other objects. This is so important and 
beneficial because these things are concrete and much easier for children to relate to, 
as opposed to just random characters that are more abstract. A child has a greater 
chance of retaining the skill or content being taught when it is presented in a 
meaningful way, that he or she can relate to, as opposed to something abstract which 
has no connection whatsoever. 



As I said, after my son had the chance to play Dragon Box, I decided to play the game 
and found it to be very addicting! If algebra had been taught this way when I was in 
middle school, I would definitely have had so much more fun and success! I love how it 
uses little avatars, fish, animals and other objects to act as the symbols (representing 
the numbers and letters). The app's development was clearly well thought out, it's very 
thorough and impressive. There are a few steps or directions that are added in here 
and there, but not on each level. Eventually, I got stuck on a level.... Which brings me 
to this - I have one suggestion that would make this app even better. Currently, there 
is no support or directions within the app itself. When the user gets stuck on a level, 
there are no hints, suggestions, or help as to what would be the next step. It basically 
just starts over and over again, keeping record of what steps worked and didn't work. 

A walk through or user guide within the app would be very beneficial. Overall, Dragon 
Box is a great app. My son really enjoyed playing it and so did I, for that matter! 
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Graphite (www.graphite.org) 



DragonBox Algebra 5+ 

Mesmerizing step-by-step puzzler sets the foundation for 
algebra 


LEARNING RATING 
TEACHER RATING 

(io Teacher Reviews) Platforms: Android, iPad, iPhone, iPod Touch, Kindle 

Fire. Windows Phone, Apps for Windows 


••••• best 

• • BEST 


Price: Paid. S5.S 

Grades: 4-8 



QinU iS 3 


Add a Field Note 
Add to a Board 


TEACHERS USING DRAGONBOX 
ALGEBRA 5+ 


£L-jl3I 


1 

i 



* Great at teaching the rules of Algebra-' 

• Heather H. 

" Algebra concepts taught initially without 
the distraction of numbers. ‘ 

- Jason J. 

" Fantastic Way to Intro Algebra ’ 

- Gustavo P. 

" Intuitive app supports algebraic 
thinking * 

• Eltsa E. 


Read 10 Teacher Reviews 


QUICK TAKE 


LEARNING SCORES 


Pros: 

A truly elegant design integrates entertainment and instruction 
so seamlessly that playing n learning. 


Engagement t) ••••• 

It's as fun to play as any best-selling game. Kids will definitely enjoy the challenge - especially as 
equations become harder and harder to balance. 


Cons: 

Some levels may be hard to solve without hints. 

Bottom Line: 

This innovative game-changer teaches algebra, changes the 
way kids learn it. and even changes how they feel about it. 

Read how we rate and review all products on Graphite 


Pedagogy f) ••••• 

Kids solve equations as they play, which makes for a wonderfully integrated experience between 
navigating the game and building critical thinking. Moving to paper may be difficult, but the 
developer's blog gives tips. 

Support <f) ••••' 

Tutorials are great for teaching rules; kids may get frustrated without any hints or help. Still, the 
website suggests adults intervene as little as possible, saying, 'Mathematics is a game. Don't spoil itr 


OUR REVIEW 

What’s It Like? 

DragonBox Algebra 5* teaches kids algebra in a refreshing and unique way. Ten chapters get increasingly complex, and drag- 
and-drop simplicity teaches kids to solve, balance, and reduce multi-variable equations and overcome fears about learning 
math. Kids get introduced to an algebraic concept with cute cartoons of baby dragons and nonintimidating language. For 
instance, the fact that integers in equations can be canceled out by their negative counterparts is called a 'night card* or 
opposite. Players must then balance equations, and they cant make a move until they put the identical card on the other 
side. As the level progresses, pictures are gradually replaced with numbers and variables, but the actions (like canceling out 
and reducing fractions and isolating X) become rote and mesmerizing. 


See Common Core Standards 


SUBJECTS 

Math 

algebra 

equations 


(continued) 
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Once the game is installed, students can customize and play with up to four avatars on the same device. Unfortunately, once 
levels are unlocked, they stay that way. so only the first students to play will get that pleasure. 

Is It Good For Learning? 

Unlike many math games. DragonBox Algebra 5 + integrates entertainment and instruction so seamlessly that learning 
gameplay is essentialy learning algebra By the time kids “win" the game, they'll be shocked by how much they've learned. It’s 
compelling because it replaces math language with the language of a game like Angry Birds. Kids will likely feel encouraged as 
they play because of the rewards system. Each level awards up to three stars: one for isolating the box (solving for X). one for 
completing the level in the right number of moves, and one for having the right number of cards. Kids have to solve levels 
correctly before they can move on to the next one They get no hints, though, so they need to figure things out themselves. 

Students will like the personalized avatars and the accessible intro tutorial that takes them through game basics step by step. 
Many kids will appreciate learning math without stale language, and teachers will definitely like how this innovative app can 
change the way students feel about algebra 

How Can Teachers Use It? 

You may consider using DragonBox Algebra 5+ in the classroom as a way to solidify concepts. Bear in mind that the full app 
buys the lesson levels. 100 bonus problems, and avatars for up to four players. Kids could theoretically share avatars, but part 
of the fun is unlocking levels. Another nice feature is the Transfer Document, which helps teachers migrate the experience from 
the mobile device to paper. The website recommends teachers be mindful that the automatic features (for example, forcing 
players to add. subtract, multiply, and divide on both sides and updating equations on one line) will need to be adapted for 
paper-and-pentil problem-solving. 

This review of DragonBox Algebra 5+ was written by Amanda Finkolberg 


Floid Notes 

Showing 6 of 9 Teacher Reviews 

Heather H. 

Austin SctXX* Honolulu 
HI 


Jason J. 

ff M FOLLOWERS 





Elisa E. 

Borxwood Moogo 
School Botfuitey. CA 



Krista M. 

f I I FOLLOWER 


□ EftcPS 

Easi Bronx Academy lor 
tho Futuro Bronx, NY 


Have you used DragonBox Algebra 5+ in your teaching? 


Add a Field Note 


Great at teaching the rules of Algebra- teacher rating 

Submitted March 4, 2014 •# • 


Algebra concepts taught initially without the teacher rating 
distraction of numbers. •• • • • 

Submitted March 4, 2014 


Fantastic Way to Intro Algebra teacher rating 

Submitted June 21,2013 • • • • • 


Intuitive app supports algebraic thinking teacher rating 

Submitted August 10,2013 


Great Game Builds Algebraic Thinking! teacher rating 

Submitted August 15, 2013 •• • • • 


Playing algebra without saying it. teacher rating 

Submitted September 1. 2013 •• • • • 


Show All Field Notes For DragonBox Algebra 5 + 


SKILLS 

Thinking & Reasoning 

logic 

solving puzzles 
strategy 

GENERAL INFORMATION 

Great For 

Individual 
Knowledge gain 
Small group 
Student-driven work 

Great With 

Advanced learners 
General 

Setup Time 

Less than 5 minutes 

Tech Notes 

Minimum requirements: <OS 4.0 or later; 
Android 2.2 and up 

Teacher Dashboard 

None 

Publisher 

WeWantToKnow AS 


Latost poll 

What would you call our 
tech-rich lesson plans (App 
Flows) if you could change 
the name? 

O App Row 
Q Tech Flow 
Q Lesson Flow 
O Flow 
Q Activity Flow 


Vote 


Older polte 
Results 
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Appitic (www.appitic.com) 


DragonBox Algebra 5+ 


$5.99 USD 



Supported Devices 


Price 



Description 



DragonBox+ Algebra - The whole algebra in a single game 


USA today: "brilliant, kids don't even know that they are doing math" 


^ J 0 ■■ 9 


Wired: "DragonBox is making me reconsider all the times I've called an educational app "innovative."" 

Forbes: "Step aside sudoku, algebra is the primordial puzzle game." 

"My son jumped right into the game. About an hour and a half later he was solving junior high level algebraic equations. 
Amazing." 

DragonBox introduces the main math elements in a playful and colorful world that appeals to children. The game features a 
gradual learning of fundamental algebraic elements (equal sign, zero, multiplication, division, parenthesis, negative numbers, 
fractions, ...). 

The player learns at his/her own pace, experimenting with rules that are introduced step by step. Progress is illustrated with the 
birth and growth of a dragon for each new chapter. 

This game primarily targets children from 6 to 16 who are willing to discover algebra or to consolidate past learning and gains. 
Playing doesn't require supervision, although parents can help making the learning more efficient and transferring learned skills 
in real life. It is a great game for parents to play along with their kids - and freshen up their own math skills. 

DragonBox is based on an original concept developed in Norway, introducing a novel pedagogical method based on discovery 
and experimentation. The game greatly helps children to make it up with math, and contributes to sharpen their desire to learn 
and to be successful with math. 

This game is the subject of extensive research at the Center for Game Science of the University of Washington. 

DragonBox has received many distinctions, and among others, the Gold Medal of the 2012 Serious Play Award (USA) and the 
Best Serious Game at the 2013 IMGA Awards. It is recommended by Common Sense Media. 


* 10 progressive chapters (5 learning, 5 training) 

* 200 puzzles 

* Suggested (and not put forward) rules the child can experiment with 

* Multiple profiles for easy progress control 

* Dedicated graphics and music for each chapter 

* 16 supported languages (English, frangais, norsk, svenska, dansk, espanol, ???, italiano, portuguSs, Deutsch, ???????, ????, 
????, suomi, nederlands, euskara) 


Features 


(continued) 
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Testimonials 

"Brilliant - My 7-year-old son has now grasped pre Algebra because of this app." 

"Unbelievably engaging for kids... - I was hoping for the best with this app and wasn't disappointed. Totally teaches algebra 
without them even realizing it. Awesome!" 

"My 12 year old can do algebra because of this program, and her 8 year old sister wants to try it." 

"This is definitely the most intuitive math learning program since Algebra Touch" 

"I'm not dumb. Fluent in 7 languages. No problems with Chemistry, physics or even Trigonometry. Just. Not. Algebra. No teacher 
or friend ever managed to get through to me. Even though I fully understood that it is more about following the rules than 
understanding.. Very, very frustrating. Gave this a try for a dare. Spent one hour playing and halfway through the bonus levels I 
was already solving them in my head.. I'm shocked and amazed. Don't know wether to laugh or cry hysterically. Thank you. A 
HUGE weight has just dropped of my shoulders. I feel a foot taller. Thank you SO much! I still can't believe it..." 

"Through this simple game, I have started to see math in a new light. Thank you for writing this excellent app!!!" 


Support 

To all users, please contact us for report issues & improvements. It's unfortunately not possible to answer your individual posts 
on the market. Thanks for your understanding ! 


SCREENSHOTS 
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Appendix 2: Technical Checklist Sample 


1. Technical 




A. Mv platform(s) is/are: 

iOS 


B. Mv device(s) is/are: 

iPad 


C. Mv operating systems is/are: 

iOS 7 




2. Pedagogical 




What is the learning objective? 

Complete algebra problems with one unknown 
variable 


To achieve that objective 1 believe 
1 need a: (underline one) 

Instructional app 
Creativity/Productivity app 
Tool app 
Book app 

3. Selection 




Instructional app 

Dragonbox Algebra 5+ 


Feedback: 

Yes 


Adapting Difficulty: 

Yes 


Mastery-Based: 

Yes 


High Response Rates: 

Yes 


Clear Learning Objectives: 

Yes 


Performance Reports: 

No 


Usability: 

Yes 


Creativity app 



Easy-to-use: 



Effective mobile layout: 



Exportable product: 



Files can be imported: 



Clear entry requirements: 



Tool app 



Information is accurate: 



Tool is intuitive/easv-to-use: 



Book app 



Reading level is age-appropriate: 



Content is age-appropriate: 


4. Special 
Considerations 




Aligned to standards? 

Yes 


Accessible? 

No 


In-app purchases? 

No 


Social media link? 

No 


Push notifications? 

No 


Advertisements? 

No 
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Appendix 3: Technical Checklist 


1. Technical 




A. Mv platform(s) is/are: 



B. Mv device(s) is/are: 



C. Mv operating svstems is/are: 





2. Pedagogical 




What is the learning objective? 



To achieve that objective 1 believe 
1 need a: (underline one) 

Instructional app 
Creativity/Productivity app 
Tool app 
Book app 

3. Selection 




Instructional app 



Feedback: 



Adapting Difficulty: 



Masterv-Based: 



High Response Rates: 



Clear Learning Objectives: 



Performance Reports: 



Usability: 



Creativity app 



Easy-to-use: 



Effective mobile layout: 



Exportable product: 



Files can be imported: 



Clear entry requirements: 



Tool app 



Information is accurate: 



Tool is intuitive/easv-to-use: 



Book app 



Reading level is age-appropriate: 



Content is age-appropriate: 


4. Special 
Considerations 




Aligned to standards? 



Accessible? 



In-app purchases? 



Social media link? 



Push notifications? 



Advertisements? 
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